1. Field of the Invention
The present invention relates to a technique for detecting faces in images.
2. Description of the Related Art
An image processing method for automatically detecting a specific object pattern from an image is very useful, and can be utilized in determination of, e.g., a human face. Such method can be used in many fields such as an online meeting, man-machine interface, security, a monitor system for tracing human faces, image compression, and the like. As a technique for detecting faces from images, for example, “Detecting faces in Images: A Survey”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 24, NO. 1, JANUARY 2002, proposes various methods. Of these methods, methods of detecting human faces by utilizing some remarkable features (two eyes, mouth, nose, and the like) and a unique geometric positional relationship among these features, symmetric features of human faces, features of skin colors of human faces, template matching, neural networks, and the like have been proposed.
As an example of the former type, “Multiresolution Face Detection”, Theoretical Foundations of Computer Vision, vol. 69, 1993, has proposed a method of detecting faces in images based on detection of blobs and streaks. That is, by utilizing the fact that eyes are dark features (dark blobs), cheekbones and a nose are light features (light blobs), and the outlines of eyebrows, a mouth, and a face are linear edges (streaks), these features are detected from an image, and a face is detected in consideration of a geometric positional relationship among these features. Also, blobs as relatively simple features are detected from a low-resolution image, and blobs and streaks are detected from a high-resolution image with high precision, thus speeding up processing. However, in such method based on feature detection, the detection results are readily influenced by changes in postures of objects, illumination conditions, noise components of images, and the like, and it may be difficult to perform detection with high precision under these conditions.
On the other hand, a method proposed by “Neural network-based face detection”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 1, JANUARY 1998, detects a face pattern in an image using neural networks. A face detection method according to this reference will be briefly described below.
Image data including a face is loaded onto a memory, and a predetermined region that matches the face is extracted from the image. Then, one output is obtained by arithmetic operations using neural networks to have the distribution of pixel values of the extracted region as an input. At this time, the weights and thresholds of the neural networks have been learned based on a huge number of face image patterns and non-face image patterns. For example, when the output of the neural networks is 0 or more, a face is determined; otherwise, a non-face is determined.
By sequentially scanning the extraction position of an image pattern to be collated with a face as the input to the neural networks from the entire image in the vertical and horizontal directions, as shown in, e.g., FIG. 3, a face is detected in the image. To cope with detection of faces with various sizes, the loaded image is sequentially reduced in size at a predetermined ratio, as shown in FIG. 3, and the above scan of face detection is applied to the reduced-scale image. Although such method based on object pattern collation can execute detection relatively robust against changes in postures of objects, illumination conditions, noise components of images, and the like, since patterns extracted from the image are collated cyclopaedically, a huge computation time is required.
To solve the above problems, a method of detecting an object by combining the advantages of the above two methods may be used. By the method according to “Multiresolution Face Detection”, Theoretical Foundations of Computer Vision, vol. 69, 1993, eye candidates are extracted from an image, and a face candidate is set to have two eyes selected from the eye candidates. Then, an image pattern is extracted from the set candidate, and face determination is made using neural networks by the method according to “Neural network-based face detection”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 1, JANUARY 1998.
FIGS. 6A to 6D show face detection based on this processing method. FIG. 6A shows a face image as a part of an input image, and positions a1 and a2 of eyes are extracted from this face image. A face region b is set based on the positional relationship between a1 and a2, and the entire face. This face region b is extracted as an image pattern shown in FIG. 6B and is collated. However, for example, when the line of sight of an object deviates from front direction, as shown in FIG. 6C, since dark blobs c1 and c2 are extracted as the eye positions. Thus, a face region is set as shown in FIG. 6C, and collation is done using an image pattern shown in FIG. 6D.
In this case, since the face positions in collation patterns are different, the image pattern is not recognized as a face pattern, and a face may not be detected normally. Note that such situation is likely to occur not only when the line of sight of an object deviates but also when, for example, the detection positions of dark parts deviate due to the influence of illumination or when eyebrows and frames of spectacles are mistaken for eyes.
Therefore, in the above prior arts, the processing for detecting an object pattern from an image cannot achieve both high precision and short computation time. Also, even in a simple combination of the above prior arts, the detection precision of an object pattern impairs.